Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Syslog ng ctl attach debugger #327

Open
wants to merge 24 commits into
base: main
Choose a base branch
from

Conversation

bazsi
Copy link
Member

@bazsi bazsi commented Oct 6, 2024

This is a continuation of #326 to add interactive debugger functionality over syslog-ng-ctl

@bazsi bazsi force-pushed the syslog-ng-ctl-attach-debugger branch 3 times, most recently from cd645a3 to acc366e Compare October 12, 2024 20:01
@bazsi bazsi changed the title WIP: Syslog ng ctl attach debugger Syslog ng ctl attach debugger Oct 13, 2024
@bazsi bazsi force-pushed the syslog-ng-ctl-attach-debugger branch from acc366e to ca8b036 Compare October 27, 2024 11:30
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
scratch_buffers_lazy_update_stats() uses the ivykis time state to check
if it is time to update the stats about scratch buffers. Do not
do that if ivykis is not initialized. This may happen in control threads and
the debugger threads that do use scratch buffers but don't use ivykis.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
A better solution would be to have a connection specific worker list,
and a list of connections. But for now this suffices for my purposes
of being able to cancel connection specific workers.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Up to now, control worker threads were only cancelled at exit. Truth be
told we never really detected if the peer has disconnected either.

This patch implements thread cancellation whenever a connection closes,
to detect the closure of the connection comes in a separate patch.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
This will be used to pass over the stdio file descriptors from syslog-ng-ctl
so we can attach to the syslog-ng process after it was started in the
background.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
This new command allows to reconnect the stdio streams even if syslog-ng
runs in the background.  syslog-ng-ctl will be able to pass 3 fds to the
syslog-ng process, which will be duplicated into the standard fds and with
that syslog-ng will happily start displaying its stdout to that terminal.

The ATTACH command itself is threaded and the control socket is only used to
indicate that the peers are still alive.  syslog-ng will start sending
"ALIVE" messages to this stream every second or so.  If sending this packet
is not successful, the connection is closed and the thread is cancelled.
Upon cancellation, the stdio fds are restored to point to /dev/null.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
…ssages

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
This is never inlined anyway, but duplicated into all modules using
log_pipe_queue(), it is then better to just have it in one location.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
log_pipe_forward_msg() is not inlined anyway, so it will just add an extra branch.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Checking the global variable pipe_single_step_hook is quite complicated as
it is addressed $rip relative, so let's filter out the config related
pipes first and remove the same check from the debugger.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
The tracer is responsible with the communication between the debugger
thread and any worker threads. Up to this point this was not cancellable,
e.g. once a breakpoint fired, the worker stopped with no way of
cancelling this.

With the addition to tracer_cancel() pending breakpoints are let go and
at the same time tracer_wait_for_breakpoint() returns as well with
a gboolean to indicate whether it was cancelled.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
The new debugger_exit() can be called from any thread, triggers the
cancellation of the tracer and then waits for the debugger thread
to finish.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
…manner

The debugger installed the pipe_single_step_hook without locking, but that
only works if we do that at startup. Since I want to be able to
start the debugger on demand, this needs to sync with any workers that
might be executing.

On x86 it might be safe to just store the pointer, but other less
forgiving architectures (e.g. ARM) may not like that. Let's do this
properly.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
…un from the main thread

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
The key ingredient to support multi-threaded debugger session is to allow
the breakpoint sites to submit debugger_stop_on_breakpoint() calls
in parallel so that these requests are queued and no variables are
used for the different invocations.

The solution is to introduce the BreakpointSite struct, this is where
we store state that relates to each distinct breakpoint events.

This struct contains both the state the debugger may want to inspect (e.g.
msg, pipe) and also  the state related to inter-thread communication
(e.g. resume_requested).

Each such site instances are queued in an internal queue which is then
processed by the interactive debugger.

Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
Signed-off-by: Balazs Scheidler <balazs.scheidler@axoflow.com>
@bazsi bazsi force-pushed the syslog-ng-ctl-attach-debugger branch from ca8b036 to e058b86 Compare October 27, 2024 11:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant